Custom texture masking issues

**twenty2man** · August 22nd, 2019, 04:50 PM

Hi All,

I've been exploring the creation of my own tree models to add into a custom map (which I may or may not ever finish), and I have been struggling with a problem which I can't seem to overcome and I'm hoping someone here may have the answer.

Every time I create a texture for the tree foliage and using a mask layer, the result is an ugly white and what seems to be a semi-transparent halo perimeter around the outside.

maskhalo.png

Now I'm not great with graphical editing software but I did some googling and I though maybe de-fringing would help but I'm getting the same thing even with blocks of colour where there is no complicated edge.

I'm saving the image as a 24 bit uncompressed TGA. I'm using the shader similar or same as the trees.shader from the pak.

Code:

static_smdmtree4_2
{
	qer_editorimage textures/models/natural/smdmtree4_2.tga
	nomipmaps
	cull none
//	deformVertexes flap t 24 sin 0 2.5 0    0.2 1 0
	deformVertexes flap t 24 sin 0 2.5 0.25 0.3 1 0
	{
		map textures/models/natural/smdmtree4_2.tga
		depthWrite
		alphaFunc GE128
		alphaGen distFade 1400 1400
		rgbGen vertex
	}
}

I make the masks by duplicating my image, then using greyscale/negative/colour replacer to get the black and white mask, loading the mask layer from this image with "source illuminance" and then saving that mask to the alpha channel before exporting.

How can I get a nice crisp edge? Any help or suggestion is appreciated.

Thanks

Twenty

**Zappa** · August 22nd, 2019, 06:54 PM

24 bit TGA supports opaque images only
32 bit TGA supports transparent images with alpha channel

so, saving it as 32 bit uncompressed should fix your problem

also, using "rgbGen vertex" may give you a error spammed in the console saying "vertex colors are not valid for this model" when testing your map with developer set to 1
you can prevent that using rgbGen lightingSpherical instead.

**1337Smithy** · August 23rd, 2019, 03:20 AM

rgbGen vertex is used for calculating static lighting values for static models baked into the map. I think that error spam only occurs if you were to spawn the object as a dynamic entity, so it's likely he'll be fine with using it. It's the same way the other static props are configured. As long as he's placing them in Radiant and compiling it shouldn't be an issue.

**twenty2man** · August 23rd, 2019, 10:55 AM

Thanks for the replies. I think I may have solved it.

Yes the models are inserted in radiant and then compiled so I'm not getting any spammed error messages in console.
My understanding was that at 32bit TGA was a 24bit with an 8 bit alpha channel and since I was only able to save up to 24bit (32bit wasn't an option) but was still able to make textures with transparencies then it should be fine as long as the alpha channel is there in the masking process. I even downloaded a trial version of PSP2020 to see if there were additional saving options and it only allows "8", "16" and "24/32" bit TGA.

Anyway for anyone else who had this issue two things I noticed. First the texture I was using was not the best quality when when I added a black background it showed up lighter edges in the texture so I manually darkened the pixels around the edge.

Before and after:

ba.png

Secondly I noticed turning on the alpha channel in radiant, the stock models all had black backgrounds and mine were white. So during the masking process, after saving the alpha channel to the image, add a new layer under the image and fill with black and merge the visible layers and then save as a TGA. This seems to make all the difference:

final.png

Looks much better!

[edit] added image with without black background layer for comparison

wbl.png

Incidentally if anyone else is still making content for Mohaa still and wants to try making their own foliage and trees, try Tree It tree generator by evolved software. You can highly customise trees and I managed to export as OBJ and import into Milkshape without too many problems - certainly good for making trunks and branches if nothing else. There another software called Tree[d] which I haven't tried but apparently does the same thing.

Twenty

**Zappa** · August 23rd, 2019, 04:02 PM

what software where you using? photoshop?
I recommend you to try GIMP. It's free and open source.
It's ultra easy to add a Layer mask to textures with it, just 2 or 3 cliks and you are done.

**twenty2man** · August 24th, 2019, 06:23 PM

I'm using an older version of paintshop pro although it does everything I need it to do still. I've downloaded gimp too but I guess I've gotten so used to knowing where everything is in PSP that I haven't used it much and I couldn't find any save options like bit size and compressed/uncompressed when I exported a TGA in gimp.

**Slimbips {sfx}** · August 26th, 2019, 03:44 AM

photoshop piratelink, photoshop is very easy to use

**AccadaccA** · November 13th, 2019, 07:37 PM

I use Paint Shop Pro too. Have done for decades.
Unfortunately it wasn't until version 5 that it could create or even edit TGA files and even then it's messy.
Try to edit it a second time and you get all sorts of distortion.
I also have PSP8 but only use that for TGAs when version 5 fails me.
PSP 8 turns your image upside down for some reason.

In the menu bar on top of PSP5 you have "Mask".
In that menu you have the option to "Save To Alpha Channel"
It won't be enabled if you still have "Edit" mode selected (in the same menu).
Save the mask to "Alpha Channel 1". You'll need to type it in if it's not already there.
Then close the image and save when prompted.

This seems to work better than manually saving before closing the image.

As a precaution it's a good idea to save a copy of the image in any other format (except horrible jpg) even as a ".PSP" file.
Just in case you need to edit it again and it screws up which TGAs often do in PaintShopPro.
Then you can paste a copy of your back-up onto the TGA.

I don't recall PSP5 having the bit size or compression options but PSP8 does.
I never touch those options.

Good luck.

**AccadaccA** · November 13th, 2019, 08:14 PM

@ twenty2man
Great tip on "Tree It tree generator".
I've just watched the video and downloaded the program.
https://80.lv/articles/tree-it-a-free-tree-generator/

Thank you.

Thread: Custom texture masking issues

View First Unread

Thread Tools

Display

Custom texture masking issues

Posting Permissions